The present invention relates generally to storage systems and, more particularly, to the decoupling of content and metadata for storage in a distributed object storage ecosystem.
With the growth of unstructured non-mutable digital content, it is becoming increasingly difficult to manage and locate relevant digital content. To find relevant content, the data about the digital content (i.e., metadata) is becoming more important than the digital content itself. For traditional object storage systems in a distributed environment (distributed object storage ecosystem), both the digital content and the metadata are stored together in multiple locations to achieve both disaster recovery and locality of reference. This is achieved by utilizing replication technology to ensure copies are distributed to remote sites. Another drawback of the traditional systems is that they apply the same rules of storage to both data and metadata. For example, if an object is stored in N copies on the low latency storage system, all N copies contain both data and metadata even though the data part might not be needed at all in some locations/applications.
Because companies are becoming more geographically dispersed with many separate offices and even data centers, the replication topologies required to provide locality of reference for all digital content becomes more complex and the storage requirements for the digital content are multiplied by the number of local references required. When the primary goal is to have locality of reference for the metadata only, this means that the digital content for the metadata is stored in multiple locations unnecessarily. Data replication is also a very time consuming procedure and replication of both data and metadata creates considerable time delays, replication backlog, and unnecessary bandwidth consumption.